Validation of content and structure of the Return-to-work assessment for post-stroke survivors

Background Validation of an instrument consist of three main types: content, criterion and construct. Content validity needs to be determined in order for an instrument to be acceptable for use, validity establishes the fact that an instrument measures exactly what it proposes to measure. The Return-to-work assessment scale (RAS) was developed to measure three aspects of return to work: (Personal factors and/or issues, work issues and contextual factors) in 2021. Objective To report on the processes followed in establishing the face and content validity of the RAS. Method Twenty participants took part in our study, they were selected purposively and conveniently from a pool of professionals and post stroke survivors. The Delphi survey technique was used to arrive at consensus and professional opinion on the items included in the RAS. Consensus was sought on the items, domains and subdomains included in the RAS that was used to assess return-to-work after a stroke. Our study was concluded after the third round. Result One item was remove out of the original 86, three (3) domains made up of eleven (11) subdomains were retained. The RAS had consensus of 100% after three rounds of scrutiny for all items. Conclusion The RAS was found to be valid, thereby establishing its face and content validity. Clinical implication The RAS is valid and was recommended for psychometric testing which was the next stage after face and content validity.


Introduction
http://www.sajp.co.za Open Access of the impact of disease on performance of everyday tasks are now commonly employed by clinicians and researchers (Ibikunle et al. 2021b:5;Muller, Roder & Greennough 2006;Sinha, Nijhawan & Grover 2014). They are usually classified as generic or disease specific. General health outcome measures are intended towards recapitulating or summarising details of the outcomes of most health conditions among patients and populations, while disease-specific outcome measures evaluate the impact of specific health conditions on the functional status of patients (Ibikunle et al. 2021b:5;Kampstra et al. 2018;Partrick 1990). These disease-specific outcome measures are observed to be more responsive to the target population when compared to generic measures (Davidson & Keating 2002;Ibikunle et al. 2021a;Kampstra et al. 2018;Muller et al. 2006).
The return-to-work assessment scale (RAS) was developed by Ibikunle et al. (2021a) as a health-specific scale for the purpose of measuring return to work among post-stroke survivors. The International Classification of Functioning, Disability and Health (ICF) and the Flag model were used for the conceptual mapping, and the theoretical framework adopted for the measure was the modified C-OAR-SE theory (Ibikunle et al. 2021a). C-OAR-SE is an acronym for the six aspects of the theory: 'C' stands for construct definition, 'OAR' for object representation, attribute classification and rater entity identification, and 'SE' for selection of item type and answer scale, as well as enumeration and scoring rules (Diamantopoulos 2005;Ibikunle et al. 2021a;Rossiter 2011a).
The three phases of development of the RAS as described by Ibikunle et al. (2021a) are (1) construct development (initial item generation), (2) face and content validation of the instrument and (3) psychometric testing. Return to work was conceptualised into three views: (1) personal factors that include grooming, independence, psychological and emotional balance; (2) work-related issues such as mobility, employees' and employers' attitudes and infrastructures and (3) contextual issues that were limited to support from family, relations, coworkers and society (company or country labour policies were not considered in our study). These three issues gave theoretical definition to RTW, each aspect answering to the capacity of the post-stroke survivor to predict returning to work, which brings independence, self-esteem and improved quality of life -the core values that can be achieved through rehabilitation. The conceptual framework was the ICF saddled with the flag model in developing the instrument using the six stages of thematic analysis after obtaining the in-depth interviews from the participants.
Many outcome measures used for the measuring of various variables among stroke patients include the Gross Motor Rivermead Assessment Scale by Lincoln and Gladman (1962), the Barthel Index by Mahoney and Barthel (1965) and the Nottingham Extended Activities of Daily Living by Finch et al. (2002), to mention but a few. Developing a health-specific outcome measure to assess return to work among stroke survivors will help to assess their readiness, as well as to monitor the RTW stages. With no means of evaluating return to work among post-stroke survivors, they will not be accepted back into their formal employment, which could affect their self-esteem, confidence and social identity (Balasooriya-Smeekens et al. 2016;Ibikunle et al. 2021b:7). Consequently, there is a need to develop an instrument that takes into consideration the contextual issues and the viewpoints of the employers and employees, as well as other important issues peculiar to post-stroke survivors, namely psychosocial and physical issues of the clime and region. Our study, which is the second phase of the research, consists of face and content validation. In this phase, consensus was sought on the items, domains and subdomains included in the RAS, which were used to assess return to work after stroke (Ibikunle et al. 2021b:8). The Delphi survey technique was deemed appropriate, as it addressed face and content validation.

Procedure
Phase 1, construct development (initial item generation), utilised in-depth interviews with a set of semistructured questions (Online Appendix 1, Interview guide) to produce transcribed responses to the questions obtained from 18 participants who were the panellists. The 18 participants were invited by the author and interviewed; they included seven post-stroke survivors, five rehabilitation specialists (one occupational therapist, three physiotherapists and one occupational nurse), three caregivers and three employers of labour. Their transcribed responses (see Online Appendix 1, Interviews on return-to-work post-stroke) were analysed using the six steps of the thematic analysis by Braun & Clarke (2006) (familiarisation, coding, generating theme, reviewing theme, defining or naming and writing up), thereby arriving at five themes which emerged from using the ATLAS.ti 7.5.0 software. The transcribed interviews were converted to PDF and uploaded into the qualitative analysis tool. Common concepts were identified, from which codes and quotations were derived, and later themes emerged which were extracted and transcribed. The five themes obtained were (1) impairment and functional limitations resulting from stroke; (2) cognitive and psychological limitations resulting from stroke; (3) barriers to RTW post-stroke; (4) facilitators of RTW post-stroke and (5) stroke as a social responsibility. The five themes were further compressed by the author and the study supervisors, who are experts in instrument development, in a focus group discussion (virtual) which produced 3 domains, 11 subdomains and 85 items. The development of a draft document, referred to as the RAS, was the first phase of the research.
The next phase, referred to as phase 2, utilised the Delphi survey technique to validate the content (items) of the scale. The Delphi technique is a useful research tool that can be used to obtain consensus from a chosen group (Gupta & Clarke 1996). The Delphi technique was selected to obtain expert opinion, as it allows for wide consultation while eliminating geographical constraints (Gupta & Clarke 1996). Experts in the field of rehabilitation were selected from academics and healthcare practitioners, as well as acute poststroke survivors who had recovered and had returned to work at the time of this survey. Twenty-five experts and stakeholders were approached to participate in the study, being 10 experts from South Africa, 10 from Nigeria and 5 post-stroke survivors from Nigeria. However, of the 25 approached, only 20 of these experts agreed to participate in our study. Four professionals from South Africa did not respond, and one declined to participate because of inadequacy of required knowledge. The professionals were to have at least between 5 and 10 years of clinical experience in neurological rehabilitation. The group which participated included 10 rehabilitation experts (eight physiotherapists and 2 occupational therapists), 3 psychologists, 2 employers and 5 post-stroke survivors. The employers had at least between 5 and 10 years' experience of recruiting workers for employment. The psychometrists were experts in the development of instruments and psychometrics. The psychologists, who had health psychometrics backgrounds, assisted with the development and finalisation of a scoring system for the newly developed instrument. Only two African countries were involved, South Africa and Nigeria. This article is the product of a thesis supervised from South Africa and carried out in Nigeria; in our study, contextual factors did not include country, company laws or policies, and this was not part of the scope of our study. Contextual factors were limited to support from family, relations, coworkers and society.

Data collection and stimulus prompt
The draft instrument was used as the stimulus prompt in the Delphi process. The RTW outcome measure consisted of two sections. Section A involved general sociodemographic issues, while section B contained the three main domains. In section A, the participants were requested, on the Delphi form, to state how relevant each question in the general section was to their respective subheadings. They were requested to answer yes or no -yes, if relevant, and no, if not. In addition, they were invited to suggest any other question that they deemed relevant, if not already included in the scale.

Delphi survey technique -Rounds 1 -3
A three-round e-Delphi technique was employed. The objectives of these rounds were to establish face and content validity, obtain consensus regarding items to be included in the outcome measure, assess return to work among poststoke survivors and develop the overall scoring system. The draft measure was revised during the Delphi rounds. The revised measure was used in phase 3 to establish the psychometric properties.

Delphi technique round 1
This was the first round; the preliminary instrument which was developed was e-mailed to experts and participants for three weeks, with e-mails sent at intervals to monitor compliance to request.

Delphi technique round 2
Item reduction was the purpose of this second stage. Any item that did not obtain a consensus of 100% from experts and participants was resent for the purpose of complete consensus.

Delphi technique round 3
In this third and last round, the third draft with all corrections effected was sent to the experts and participants. The health psychologists were involved from the start, especially regarding issues related to the scoring system. All participants accepted the scoring system as presented by the authors and perceived no need for any change. The new instrument had to cover the domains that constitute return to work among post-stroke survivors. The level of consensus reached on the suitability and appropriateness of each item included in the scale at the end of the Delphi study was 100%.
There is no consistent reference available to determine the standard level of consensus. The three rounds of the Delphi technique were hence used to facilitate reaching consensus; the percentage set for consensus was 100%. (Ibikunle et al. 2021b:48)

Time requirements
Two weeks were given as the minimum time for the experts and participants to respond; we adopted 6 weeks as the maximum time for each round of the Delphi process.

Ethical considerations
Ethics approval and project registration were sought and obtained from the Senate Research Committee of the University of the Western Cape (reference number: 15/2/20). In Nigeria, ethics approval was also sought from the Faculty of Health Sciences and Technology Ethics Committee of Nnamdi Azikiwe University (reference number: ERC/FHST/NAU/2018/028).

Results of round 1 Section A of the return-to-work scale
This section is made up of five subsections, including demographic information; type, area and severity; impairments or defects; post-stroke management and nature of employment (see Online Appendix 1, Return-to-work assessment scale).

Section B of the return-to-work scale
This section contains the 3 domains, 11 subdomains and 86 items that were endorsed to assess return to work among post-stroke survivors. These, along with their scoring patterns, are presented in Tables 2-7.

Results of round 2
Because of the high consensus level in round 1, round 2 merely served to reach total consensus in areas where 100% agreement was not obtained. This was resent to the participants to ensure total consensus.

Section A of return-to-work scale
This section contains the result of section A of the RAS after round 2 of the Delphi Study, here the consensus among participants is now 100% (see Table 6).

Section B of return-to-work scale
This section contains the three domains of the RTW scale. Two domains reached 100% consensus from the experts and patients. However, in domain 2, subdomains 2.1 and 2.3 were resent after the changes were effected to ensure 100% consensus.

Domain 2: Work
This section refers to Domain 2, here final consensus and agreement was reached by the participants (see Table 7).

Results of round 3
The third round of Delphi was conducted to achieve final consensus, which materialised when every participant  agreed to all the sections, as well as the various domains, after the suggested changes were implemented. The scoring system was assessed along with the study and agreed upon by the 20 experts. A consensus of 100% was obtained in all items, domains and subdomains.

Discussion
The aim of our study was to report the process followed in establishing the face and content validity of the RAS while adopting the Delphi survey technique. A draft document referred to as RAS was developed by Ibikunle et al. (2021a); its face and content validity were determined and its psychometric properties were also established. The scale was found to be reliable and structurally valid. The result of the face and content validity as reported here revealed a consensus of 100% for all items, domains and subdomains that made up the RAS among the participants of the Delphi survey, indicating good face and content validity. This is in agreement with the works of Glassel et al. (2011);Sullivan et al. (2012) and Yu et al. (2013), which also adopted the use of a Delphi survey study in establishing face and content validity as suitable and sufficient to declare validity of a newly developed instrument. The authors placed their emphasis on the high content validity (100%) of the items and answer scale or answer scales; these arguments and definitions posit content validity as necessary for reliability, reversing the usual psychometric argument that reliability is necessary for validity (Rossiter 2012). There are some RTW studies among stroke survivors carried out in South Africa, Nigeria, United Kingdom and United States of America; none of them set out to develop an instrument for assessing return to work for post-stroke survivors (Alaszewski et al. 2007;Black-Schaffer & Osberg 1990;Busch 2009;Gilworth et al. 2009;Kauranen et al. 2013;Lock et al. 2005;Obembe et al. 2010;

Limitation
Country and company policies and laws were not part of the contextual factors studied in this study.

Conclusion
It can be concluded from the result of the study that the RAS has good face and content validity.

3.
Impairments or defects 100 20/20 Suggestions were made to break down the questions and simplify defects. The suggestion was implemented by breaking defects down to speech and walking impediments, and after this inclusion, consensus reached 100%.

4.
Post-stroke management 100 20/20 There was a suggestion to break down the level of hospitalisation for improved understanding. The suggestion was implemented by including level and length in hospitalisation, after which consensus level reached 100%.

5.
Nature of employment 100 20/20 Suggestions were made to redefine the type of employment. The suggestion was implemented by elaborating employment into temporary, casual, contract or permanent, and the consensus level reached 100%.